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The  Honorable  Carl  Levin 
Chairman 

The  Honorable  John  Warner 
Ranking  Minority  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Bob  Stump 
Chairman 

The  Honorable  Ike  Skelton 
Ranking  Minority  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Defense  Logistics  Agency  (DLA)  plays  a  critical  role  in  supporting 
America’s  military  forces  worldwide.  To  fulfill  this  role,  DLA  employs 
about  28,000  civilian  and  military  workers,  located  at  about  500  sites  in  all 
50  states  and  in  28  countries;  in  round  numbers,  it  manages  4  million 
supply  items  and  processes  30  million  annual  supply  distribution  actions. 

In  fiscal  year  2001,  DLA  reported  that  these  operations  resulted  in  sales  to 
the  military  services  of  about  $15.2  billion.  To  perform  its  logistics  support 
mission,  the  agency  relies  extensively  on  information  technology  (IT).  For 
fiscal  year  2002,  DLA’s  IT  budget  is  about  $654  million. 

This  report  is  one  in  a  series  of  products  to  respond  to  the  fiscal  year  2001 
Defense  Authorization  Act.'  That  act  directs  that  GAO  review  DLA’s 
efficiency  and  effectiveness  in  meeting  customer  requirements,  application 
of  best  business  practices,  and  opportunities  for  improving  agency 
operations. 

This  report  focuses  on  DLA’s  processes  for  making  informed  IT 
investment  decisions.  As  agreed  with  your  offices,  our  objectives  were  to 
determine  (1)  whether  DLA  has  effective  IT  investment  management 
processes  needed  to  modernize  and  maintain  systems  and  (2)  what  actions 
the  agency  has  planned  to  improve  these  processes. 


'P.L.  106-398,  Floyd  D.  Spence  National  Defense  Authorization  Act  for  Fiscal  Year  2001, 
section  917. 
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Results  in  Brief 


Because  IT  investment  management  has  only  recently  become  an  area  of 
management  focus  and  commitment  at  DLA,  the  agency’s  capability  to 
effectively  manage  its  IT  investments  is  limited.  DLA  has  recognized  the 
need  to  strengthen  its  processes  in  this  area.  For  example,  the  agency 
recently  began  introducing  basic  project  selection  and  control  activities 
into  its  longstanding  budget-driven,  decisionmaking  process.  Nevertheless, 
DLA  has  much  more  to  accomplish.  Until  DLA  fully  implements  an 
effective  IT  investment  management  process,  it  will  not  know  whether  its 
mix  of  investments  best  meets  its  mission  and  business  priorities. 

•  The  first  step  toward  establishing  effective  investment  management  is 
putting  in  place  foundational,  project-level  control  and  selection 
processes.  These  foundational  processes  allow  the  agency  to  identify 
variances  in  project  cost,  schedule,  and  performance  expectations;  to  take 
corrective  action,  if  appropriate;  and  to  make  informed,  project-specific 
selection  decisions.  Although  DLA  has  made  progress  toward  establishing 
such  foundational  processes,  key  practices  still  need  to  be  implemented. 
For  example,  the  business  needs  for  IT  projects  are  not  always  clearly 
identified  and  defined,  an  IT  investment  selection  process  has  not  been 
fully  implemented,  and  policies  and  procedures  for  project  oversight  are 
not  documented.  With  such  weaknesses,  executives  cannot  be  assured 
that  they  are  consistently  selecting  and  managing  IT  investments  that  meet 
agency  needs  and  priorities. 

•  The  second  major  step  toward  effective  investment  management  is  to 
continually  assess  proposed  and  ongoing  projects  as  an  integrated  and 
competing  set  of  investment  options.  This  portfolio  management  approach 
enables  the  organization  to  consider  the  relative  costs,  benefits,  and  risks 
of  new  and  previously  funded  investments  and  thereby  identify  the  mix 
that  best  meets  its  mission,  strategies,  and  goals.  DLA  officials 
acknowledge  that  the  agency  has  not  implemented  the  processes 
associated  with  managing  investments  as  a  complete  portfolio  (that  is,  an 
integrated,  enterprisewide  collection  of  investments).  As  a  result,  DLA 
executives  are  unable  to  adequately  assess  the  relative  merits  of 
investment  proposals  and  make  trade-offs  among  options. 

Accomplishing  these  two  major  steps  requires  effective  development  and 
implementation  of  a  plan,  supported  by  senior  management,  which  defines 
and  prioritizes  investment  process  improvements.  DLA  does  not  have  such 
a  plan.  Without  a  well-defined  process  improvement  plan  and  controls  for 
implementing  it,  it  is  unlikely  that  the  agency  will  establish  a  mature 
investment  management  capability.  As  a  result,  DLA  will  continue  to  be 
challenged  in  its  ability  to  make  informed  and  prudent  investment 
decisions  in  managing  its  annual  multimillion  dollar  IT  budget. 
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To  strengthen  DLA’s  investment  management  capability,  we  are 
recommending  that  DLA  implement  and  develop  a  plan  aimed  at 
addressing  the  weaknesses  discussed  in  this  report. 

In  commenting  on  a  draft  of  this  report,  DOD  concurred  with  our 
recommendations  and  described  efforts  under  way  and  planned  to 
implement  them.  Additionally,  it  recommended  that  two  report  captions 
be  changed  to  more  accurately  reflect,  in  DOD’s  view,  the  contents  of  the 
report  and  to  eliminate  false  impressions.  We  do  not  agree.  Both  of  the 
captions  cited  are  fully  consistent  with  the  evidence  presented  in  the 
report,  and  thus  are  completely  accurate  and  appropriate. 


Background 


DLA  is  DOD’s  logistics  manager  for  all  departmental  consumable  items 
and  some  repair  parts.  Its  primary  business  function  is  materiel 
management:  providing  supply  support  to  sustain  military  operations  and 
readiness.  In  addition,  DLA  performs  five  other  supply-related  business 
functions:  distributing  materiel  from  DLA  and  service-owned  inventories, 
purchasing  fuels  for  DOD  and  the  U.S.  government,  storing  strategic 
materiel,  marketing  surplus  DOD  materiel  for  reuse  and  disposal,  and 
providing  numerous  information  services,  such  as  item  cataloging,  for 
DOD  and  the  U.S.  government,  as  well  as  selected  foreign  governments. 
These  six  business  functions  are  managed  by  field  commands  that  report 
to  and  support  the  agency’s  central  command  authority. 

In  2000,  DLA  refocused  its  logistics  mission  from  that  of  a  supplier  of 
materiel  to  a  manager  of  supply  chain  relationships.  To  support  this 
transition,  the  agency  developed  a  strategic  plan  (known  as  DLA  21)  to 
reengineer  and  modernize  its  operations.2  Among  the  goals  of  DLA  21  are 
to  optimize  inventories,  improve  efficiency,  increase  effectiveness  through 
organizational  redesign,  reduce  inventories,  and  modernize  business 
systems. 

DLA  relies  on  over  650  systems  to  support  warfighters  by  allowing  access 
to  global  inventories.  Whether  it  is  ensuring  that  there  is  enough  fuel  to 
service  an  aircraft  fleet,  providing  sufficient  medical  supplies  to  protect 
and  treat  military  personnel,  or  supplying  ample  food  rations  to  our 
soldiers  on  the  frontlines,  information  technology  plays  a  key  role  in 
ensuring  that  Defense  Department  agencies  are  prepared  for  their 


2Defense  Logistics  Agency,  Strategic  Plan  2000:  DLA  21  (September  1999). 
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missions.  Because  of  its  heavy  reliance  on  IT  to  accomplish  its  mission, 
DLA  invests  extensively  in  this  area.  For  fiscal  year  2002,  DLA’s  IT  budget 
is  about  $654  million. 


Prior  Reviews  Identified  IT 
Management  Weaknesses 


Our  recent  reviews  of  DLA’s  IT  management  have  identified  weaknesses  in 
such  important  areas  as  enterprise  architecture  management,  incremental 
investment  management,  and  software  acquisition  management. 


In  June  2001,  we  reported  that  DLA  did  not  have  an  enterprise  architecture 
to  guide  the  agency’s  investment  in  its  Business  Systems  Modernization 
(BSM)  project — the  agency’s  largest  IT  project.3  The  use  of  an  enterprise 
architecture,  which  describes  an  organization’s  mode  of  operation  in 
useful  models,  diagrams,  and  narrative,  is  required  by  the  OMB  guidance 
that  implements  the  Clinger-Cohen  Act  of  19964  and  is  a  commercial  best 
practice.  Such  a  “blueprint”  can  help  clarify  and  optimize  the 
dependencies  and  relationships  among  an  agency’s  business  operations 
and  the  IT  infrastructure  and  applications  supporting  them.  An  effective 
architecture  describes  both  the  environment  as  it  is  and  the  target 
environment  that  an  organization  is  aiming  for  (as  well  as  a  plan  for  the 
transition  from  one  to  the  other).  We  concluded  that  without  this 
architecture,  DLA  will  be  challenged  in  its  efforts  to  successfully  acquire 
and  implement  BSM. 


Further,  we  reported  that  DLA  was  not  managing  its  investment  in  BSM  in 
an  incremental  manner,  as  required  by  the  Clinger-Cohen  Act  of  1996  and 
OMB  guidance  and  in  accordance  with  best  commercial  practices.  An 
incremental  approach  to  investment  helps  to  minimize  the  risk  associated 
with  such  large-scale  projects  as  BSM.  Accordingly,  we  recommended  that 
DLA  make  the  development,  implementation,  and  maintenance  of  an 
enterprise  architecture  an  agency  priority  and  take  steps  to  incrementally 
justify  and  validate  its  investment  in  BSM.  According  to  DLA  officials,  the 
agency  is  addressing  these  issues. 


^Further  details  on  BSM  are  provided  in  the  section  on  scope  and  methodology. 

4The  Clinger-Cohen  Act  of  1996  was  enacted  to  address  longstanding  problems  related  to 
federal  IT  management  Among  other  things,  it  requires  agency  heads  to  implement  a 
process  for  maximizing  the  value  and  assessing  and  managing  the  risks  of  its  acquisitions. 
A  key  goal  of  the  Clinger-Cohen  Act  is  that  agencies  have  processes  and  information  in 
place  to  help  ensure  that  IT  projects  are  being  implemented  at  acceptable  costs,  within 
reasonable  and  expected  time  frames,  and  are  contributing  to  tangible,  observable 
improvements  in  mission  performance. 
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In  January  2002,  we  reported  a  wide  disparity  in  the  rigor  and  discipline  of 
software  acquisition  processes  between  two  DLA  systems.5  Such 
inconsistency  in  processes  for  acquiring  software  (the  most  costly  and 
complex  component  of  systems)  can  lead  to  the  acquisition  of  systems 
that  do  not  meet  the  information  needs  of  management  and  staff,  do  not 
provide  support  for  necessary  programs  and  operations,  and  cost  more 
and  take  longer  than  expected  to  complete. 

We  also  reported  that  DLA  did  not  have  a  software  process-improvement 
program  in  place  to  effectively  strengthen  its  corporate  software 
acquisition  processes,  having  eliminated  the  program  in  1998.  Without  a 
management-supported  software  process-improvement  program,  it  is 
unlikely  that  DLA  can  effectively  improve  its  institutional  software 
acquisition  capabilities,  which  in  turn  means  that  the  agency’s  software 
projects  will  be  at  risk  of  not  delivering  promised  capabilities  on  time  and 
within  budget.  Accordingly,  we  recommended  that  DLA  institute  a 
software  process-improvement  program  and  correct  the  software 
acquisition  process  weaknesses  that  we  identified.  According  to  DLA 
officials,  the  agency  is  addressing  each  of  these  issues. 


Information  Technology 
Investment  Management 
(ITIM)  Maturity 
Framework 


In  May  2000,  we  issued  the  Information  Technology  Investment 
Management  (ITIM)  maturity  framework,6  which  identifies  critical 
processes  for  successful  IT  investment  and  organizes  these  processes  into 
an  assessment  framework  comprising  five  stages  of  maturity.  This 
framework  supports  the  fundamental  requirements  of  the  Clinger-Cohen 
Act  of  1996,  which  requires  IT  investment  and  capital  planning  processes 
and  performance  measurement.  Additionally,  ITIM  can  provide  a  useful 
roadmap  for  agencies  when  they  are  implementing  specific,  fundamental 
IT  capital  planning  and  investment  management  practices.  The  federal 
Chief  Information  Officers  Council  has  favorably  reviewed  the  framework, 
and  it  is  also  being  used  by  a  number  of  executive  agencies  and 
organizations  for  designing  related  policies  and  procedures  and  self-led  or 
contractor-based  assessments. 


5U.S.  General  Accounting  Office,  Information  Technology:  Inconsistent  Software 
Acquisition  Processes  at  the  Defense  Logistics  Agency  Increase  Project  Risks ,  GAO-02-9 
(Washington,  D.C.:  January  10,  2002). 

6U.S.  General  Accounting  Office,  Information  Technology  Investment  Management:  A 
Framework  for  Assessing  and  Improving  Process  Maturity ,  Exposure  Draft, 

G AO/AIMD- 10-1.23  (Washington,  D.C.:  May  2000). 
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ITIM  establishes  a  hierarchical  set  of  five  different  maturity  stages.  Each 
stage  builds  upon  the  lower  stages  and  represents  increased  capabilities 
toward  achieving  both  stable  and  effective  (and  thus  mature)  IT 
investment  management  processes.  Except  for  the  first  stage — which 
largely  reflects  ad  hoc,  undefined,  and  undisciplined  decision  and 
oversight  processes — each  maturity  stage  is  composed  of  critical 
processes  essential  to  satisfy  the  requirements  of  that  stage.  These  critical 
processes  are  defined  by  core  elements  that  include  organizational 
commitment  (for  example,  policies  and  procedures),  prerequisites  (for 
example,  resource  allocation),  and  activities  (for  example,  implementing 
procedures).  Each  core  element  is  composed  of  a  number  of  key  practices. 
Key  practices  are  the  specific  tasks  and  conditions  that  must  be  in  place 
for  an  organization  to  effectively  implement  the  necessary  critical 
processes. 

Figure  1  shows  the  five  ITEM  stages  and  a  brief  description  of  each  stage. 


Figure  1 :  The  Five  Stages  of  Maturity  within  ITIM 


Enterprise 
and  strategic 
focus 


Project¬ 

centric 


Maturity 


Description 


Investment  benchmarking  and  IT-enabled  change 
management  techniques  are  deployed  to  strategically 
shape  business  outcomes, 


Process  evaluation  techniques  focus  on  improving  the 
performance  and  management  of  the  organization's  IT 
investment  portfolio. 


Comprehensive  IT  investment  portfolio  selection  and 
control  techniques  are  in  place  that  incorporate  benefit 
and  risk  criteria  linked  to  mission  goals  and  strategies. 


Repeatable  investment  control  techniques  are  in  place, 
and  the  key  foundation  capabilities  have  been 
implemented  focusing  on  cost  and  schedule  activities. 


There  is  little  awareness  of  investment  management 
techniques.  IT  management  processes  are  ad  hoc, 
project-centric,  and  have  widely  variable  outcomes. 


Source:  GAO. 


Page  6 


GAO-02-314  DLA’s  Investment  Management  Process 


Scope  and 
Methodology 


Using  ITIM,  we  assessed  the  extent  to  which  DLA  satisfied  the  five  critical 
processes  in  stage  2  of  the  framework.  Based  on  DLA’s  acknowledgment 
that  it  had  not  executed  any  of  the  key  practices  in  stage  3,  we  did  not 
independently  assess  the  agency’s  capabilities  in  this  stage  or  stages  4  and 
5.  To  determine  whether  DLA  had  implemented  the  stage  2  critical 
processes,  we  compared  relevant  DLA  policies,  procedures,  guidance,  and 
documentation  associated  with  investment  management  activities  to  the 
key  practices  and  critical  processes  in  ITIM.  We  rated  the  key  practices  as 
“executed”  based  on  whether  the  agency  demonstrated  (by  providing 
evidence  of  performance)  that  it  had  met  the  criteria  of  the  key  practice.  A 
key  practice  was  rated  as  “not  executed”  when  we  found  insufficient 
evidence  of  a  practice  during  the  review,  or  when  we  determined  that 
there  were  significant  weaknesses  in  DLA’s  execution  of  the  key  practice. 

As  part  of  our  analysis,  we  selected  four  IT  projects  as  case  studies  to 
verify  application  of  the  critical  processes  and  practices.  We  selected 
projects  that  (1)  supported  different  DLA  business  areas  (such  as  materiel 
management),  (2)  were  in  different  lifecycle  phases  (for  example, 
requirements  definition,  design,  operations  and  maintenance), 

(3)  represented  different  levels  of  risk  (such  as  low  or  medium)  as 
designated  by  the  agency,  and  (4)  included  at  least  one  investment  that 
required  funding  approval  by  a  DOD  authority  outside  of  DLA  (for 
example,  the  Office  of  the  Secretary  of  Defense  (OSD)).  The  four  projects 
are  the  following: 

•  Business  Systems  Modernization:  This  system,  which  supports  DLA’s 
materiel  management  business  area,  is  in  the  concept  demonstration 
phase  of  development.  DLA  reported  that  it  spent  about  $136  million  on 
this  system  in  fiscal  year  2001,  and  it  has  budgeted  about  $133  million  for 
fiscal  year  2002.  BSM  is  intended  to  modernize  DLA’s  materiel 
management  business  function,  replacing  two  of  its  standard  systems  (the 
Standard  Automated  Materiel  Management  System  and  the  Defense 
Integrated  Subsistence  Management  System).  The  project  is  also  intended 
to  enable  the  agency  to  reengineer  its  logistics  practices  to  reflect  best 
commercial  business  practices.  For  example,  in  support  of  DLA’s  goal  of 
reducing  its  role  as  a  provider  and  manager  of  materiel  and  increasing  its 
role  as  a  manager  of  supply  chain  relationships,  BSM  is  to  help  link 
customers  with  appropriate  suppliers  and  to  incorporate  commercial 
business  practices  regarding  physical  distribution  and  financial 
management.  The  agency  has  classified  this  project  as  high  risk,  and  OSD 
has  funding  approval  authority  for  this  project. 

•  Hazardous  Materials  Information  System  (HMIS):  This  system,  which 
supports  DLA’s  logistics  operations  function,  was  implemented  in  1978.  In 
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fiscal  year  2001,  DLA  reported  that  it  spent  about  $1  million  on  this  system 
and  budgeted  about  $2.4  million  for  fiscal  year  2002.  In  1999  DLA  began  a 
redesign  effort  to  transform  HMIS  into  a  Web-based  system  with  a  direct 
interface  to  the  manufacturers  and  suppliers  of  hazardous  material.  The 
project  is  in  the  development  stage.  It  contains  data  on  the  chemical 
composition  of  materials  classified  as  “hazardous”  for  the  purposes  of 
usage,  storage,  and  transportation.  The  system  is  used  by  Emergency 
Response  Teams  whenever  a  spill  or  accident  occurs  involving  hazardous 
materials.  The  agency  classified  this  project  as  low  risk,  and  funding 
approval  occurs  within  DLA. 

•  The  Defense  Reutilization  and  Marketing  Automated  Information  System 
(DAISY):  This  system,  which  supports  DLA’s  materiel  reuse  and  disposal 
mission,  is  in  the  operations  and  maintenance  lifecycle  phase.  The  agency 
reported  that  it  spent  approximately  $4.4  million  on  DAISY  in  fiscal  year 
2001,  and  it  has  budgeted  about  $7  million  for  fiscal  year  2002.  This  system 
is  a  repository  for  transactions  involving  the  reutilization,  transfer, 
donation,  sale,  or  ultimate  disposal  of  excess  personal  property  from  DOD, 
federal,  and  state  agencies.  The  excess  property  includes  spare  and  repair 
parts,  scrap  and  recyclable  material,  precious  metals  recovery,  hazardous 
material,  and  hazardous  waste  disposal.  Operated  by  the  Defense 
Reutilization  and  Marketing  Service,  the  system  is  used  at  190  locations 
worldwide.  The  agency  classified  this  project  as  low  risk,  and  funding 
approval  occurs  within  DLA. 

•  Standard  Automated  Materiel  Management  System  (SAMMS):  This  system, 
which  supports  DLA’s  materiel  management  business  area,  is  30  years  old 
and  approaching  the  end  of  its  useful  life.  The  agency  reports  that 
investment  in  SAMMS  (budgeted  at  approximately  $19  million  for  fiscal 
year  2002)  is  directed  toward  keeping  the  system  operating  until  its 
replacement,  BSM,  becomes  fully  operational  (scheduled  for  fiscal  year 
2005).  This  system  provides  the  Inventory  Control  Points  with  information 
regarding  stock  levels,  as  well  as  with  the  capabilities  required  for 

(1)  acquisition  and  management  of  wholesale  consumable  items,  (2)  direct 
support  for  processing  requisitions,  (3)  forecasting  of  requirements, 

(4)  generation  of  purchase  requests,  (5)  maintenance  of  technical  data, 

(6)  financial  management,  (7)  identification  of  items,  and  (8)  asset 
visibility.  The  agency  has  classified  the  maintenance  of  SAMMS  as  a  low 
risk  effort,  and  funding  approval  occurs  within  DLA. 

For  these  projects,  we  reviewed  project  management  documentation,  such 
as  mission  needs  statements,  project  plans,  and  status  reports.  We  also 
analyzed  charters  and  meeting  minutes  for  DLA  oversight  boards,  DLA’s 
draft  Automated  Information  System  Emerging  Program  Life  Management 
(LCM)  Review  and  Milestone  Approval  Directive  and  Portfolio 
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Management  and  Oversight  Directives,  and  DOD’s  5000  series  guidance  on 
systems  acquisition.7  In  addition,  we  reviewed  documentation  related  to 
the  agency’s  self-assessment  of  its  IT  investment  operations. 

To  supplement  our  document  reviews,  we  interviewed  senior  DLA 
officials,  including  the  vice  director  (who  sits  on  the  Corporate  Board, 
DLA’s  highest  level  investment  decisionmaking  body),  the  chief 
information  officer  (CIO),  the  chief  financial  officer,  and  oversight  board 
members.  We  also  interviewed  the  program  managers  of  our  four  case 
study  projects,  as  well  as  officials  responsible  for  managing  the  IT 
investment  process 8  and  other  staff  within  Information  Operations. 

To  determine  what  actions  DLA  has  taken  to  improve  its  IT  investment 
management  processes,  we  interviewed  the  CIO  and  officials  of  the  Policy, 
Plans,  and  Assessments  and  the  program  executive  officer  (PEO) 
operations  groups  within  the  Information  Operations  Directorate.  These 
groups  are  primarily  responsible  for  implementing  investment 
management  process  improvements.  We  also  reviewed  a  draft  list  of  IT 
investment  management  improvement  tasks. 

We  conducted  our  work  at  DLA  headquarters  in  Fort  Belvoir,  Virginia, 
from  June  2001  through  January  2002,  in  accordance  with  generally 
accepted  government  auditing  standards. 


DLA’s  Capabilities  to 
Effectively  Manage  IT 
Investments  Are 
Limited 


In  order  to  have  the  capabilities  to  effectively  manage  IT  investments,  an 
agency  should  (1)  have  basic,  project-level  control  and  selection  practices 
in  place  and  (2)  manage  its  projects  as  a  portfolio  of  investments,  treating 
them  as  an  integrated  package  of  competing  investment  options  and 
pursuing  those  that  best  meet  the  strategic  goals,  objectives,  and  mission 
of  the  agency. 


7The  5000  series  on  systems  acquisition  includes  three  directives:  The  Defense  Acquisition 
System ,  5000.1  (October  23,  2000);  Operation  of  the  Defense  Acquisition  System,  5000.2 
(January  4,  2001);  and  Mandatory  Procedures  for  Major  Defense  Acquisition  Programs 
and  Major  Automated  Information  System  Acquisition  Programs ,  5000. 2-R  (June  2001). 

8These  officials  include  the  enterprise  portfolio  manager,  portfolio  managers,  and  staff  of 
the  program  executive  officer.  The  enterprise  portfolio  manager  and  the  individual 
portfolio  managers  are  responsible  for  overseeing  and  managing  DLA’s  various  groups  of 
systems,  which  the  agency  refers  to  as  “portfolios.”  The  PEO  is  the  senior  official  who  has 
responsibility  for  directing  major  IT  acquisition  programs. 
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Many  Stage  2  Strengths, 
but  Key  Weaknesses  Exist 
in  Project-Level  Control 
and  Selection  Processes 


DLA  has  a  majority  of  the  project-level  practices  in  place.  However,  it  is 
missing  several  crucial  practices,  and  it  is  not  performing  portfolio-based 
investment  management.  According  to  the  CIO,  the  evolving  state  of  its 
investment  management  capabilities  is  the  result  of  agency  leadership’s 
recently  viewing  IT  investment  management  as  an  area  of  management 
focus  and  priority.  Without  having  crucial  processes  and  related  practices 
in  place,  DLA  lacks  essential  management  controls  over  its  sizable  IT 
investments. 


At  ITIM  stage  2  maturity,  an  organization  has  attained  repeatable, 
successful  IT  project-level  investment  control  processes  and  basic 
selection  processes.  Through  these  processes,  the  organization  can 
identify  expectation  gaps  early  and  take  appropriate  steps  to  address 
them.  According  to  ITEM,  critical  processes  at  stage  2  include  (1)  defining 
investment  board9  operations,  (2)  collecting  information  about  existing 
investments,  (3)  developing  project-level  investment  control  processes, 

(4)  identifying  the  business  needs  for  each  IT  project,  and  (5)  developing  a 
basic  process  for  selecting  new  IT  proposals.  Table  1  discusses  the 
purpose  for  each  of  the  stage  2  critical  processes. 


Table  1:  Stage  2  Critical  Processes:  Building  the  Investment  Foundation 

Critical  process 

Purpose 

IT  investment  board 
operation 

To  define  and  establish  the  governing  board(s)  responsible 
for  selectinq,  controlling,  and  evaluating  IT  investments 

IT  project  identification 

To  create  and  maintain  an  IT  project  inventory  to  assist  in 
managerial  decisionmaking 

IT  project  oversight 

To  regularly  determine  each  IT  project’s  progress  toward  cost 
and  schedule  milestones,  using  established  criteria,  and  to 
take  corrective  actions  when  milestones  are  not  achieved 

Business  needs 
identification  for  IT  projects 

To  ensure  that  each  IT  project  supports  the  organization’s 
business  needs  and  meets  users’  needs 

Proposal  selection 

To  ensure  that  an  established,  structured  process  is  used  to 
select  new  IT  proposals 

Source:  GAO. 


To  its  credit,  DLA  has  put  in  place  about  75  percent  of  the  key  practices 
associated  with  stage  2  critical  processes.  For  example,  DLA  has  oversight 


9An  investment  board  is  a  decisionmaking  body  made  up  of  senior  program,  financial,  and 
information  managers  that  is  responsible  for  making  decisions  about  investments  or 
projects. 
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Boards  Established,  but 
Policies  and  Procedures  to 
Guide  Board  Operations  Are 
Lacking 


boards  to  perform  investment  management  functions,  and  it  has  basic 
project-level  control  processes  to  help  ensure  that  IT  projects  are  meeting 
cost  and  schedule  expectations. 

However,  DLA  has  not  executed  several  crucial  stage  2  investment 
practices.  For  example,  the  business  needs  for  IT  projects  are  not  always 
clearly  identified  and  defined,  basic  investment  selection  processes  are 
still  being  developed,  and  policies  and  procedures  for  project  oversight  are 
not  documented.  Table  2  summarizes  the  status  of  DLA’s  stage  2  critical 
processes,  showing  how  many  associated  key  practices  the  agency  has 
executed.  DLA’s  actions  in  each  of  the  critical  processes  are  discussed  in 
the  sections  that  follow. 


Table  2:  Status  of  Stage  2  Critical  Processes 

Critical  process 

Key  practices 
executed 

Key  practices  not 
executed 

Overall  total 

1 .  IT  investment  board  operation 

4 

2 

6 

2.  IT  project  identification 

5 

2 

7 

3.  IT  project  oversight 

9 

2 

11 

4.  Business  needs  identification 

7 

1 

8 

5.  Proposal  selection 

4 

2 

6 

Totals 

29 

9 

38 

Source:  GAO. 


To  help  ensure  executive  management  accountability  for  IT  capital 
planning  and  investment  decisions,  an  organization  should  establish  a 
governing  board  or  boards  responsible  for  selecting,  controlling,  and 
evaluating  IT  investments.  According  to  ITIM,  effective  IT  investment 
board  operations  require,  among  other  things,  that  (1)  board  membership 
have  both  IT  and  business  knowledge,  (2)  board  members  understand  the 
investment  board’s  policies  and  procedures  and  exhibit  core  competencies 
in  using  the  agency’s  IT  investment  policies  and  procedures,  (3)  the 
organization’s  executives  and  line  managers  support  and  carry  out  board 
decisions,  (4)  the  organization  create  organization-specific  process 
guidance  that  includes  policies  and  procedures  to  direct  the  board’s 
operations,  and  (5)  the  investment  board  operate  according  to  written 
policies  and  procedures.  (The  full  list  of  key  practices  is  provided  in 
table  3.) 
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DLA  has  established  several  oversight  boards  that  perform  IT  investment 
management  functions.10  These  boards  include  the  following: 

•  The  DLA  Investment  Council,  which  is  intended  to  review,  evaluate,  and 
approve  new  IT  and  non-IT  investments  between  $100,000  and  $1,000,000. 

•  The  Program  Executive  Officer  Review  Board,  which  is  intended  to  review 
and  approve  the  implementation  of  IT  investments  that  are  budgeted  for 
over  $25  million  in  all  or  over  $5  million  in  any  one  year. 

•  The  Corporate  Board,  which  is  intended  to  review,  evaluate,  and  approve 
all  IT  and  non-IT  investments  over  $1  million. 

DLA  is  executing  four  of  the  six  key  practices  needed  for  these  boards  to 
operate  effectively.11  For  example,  the  membership  of  these  boards 
integrates  both  IT  and  business  knowledge.  In  addition,  board  members 
informed  us  of  their  understanding  of  their  board’s  informal  practices. 
Further,  according  to  IT  investment  officials,  project  managers,  and 
agency  documentation,  the  boards  have  a  process  for  ensuring  that  their 
decisions  are  supported  and  carried  out  by  organization  executives  and 
line  managers.  This  process  involves  documenting  board  decisions  in 
meeting  minutes,  assigning  staff  to  carry  out  the  decisions,  and  tracking 
the  actions  taken  on  a  regular  basis  until  the  issues  are  addressed. 

Nonetheless,  DLA  is  missing  the  key  ingredient  associated  with  two  of  the 
board  oversight  practices  that  are  needed  to  operate  effectively — 
organization-specific  guidance.  This  guidance,  which  serves  as  official 
operations  documentation,  should  (1)  clearly  define  the  roles  of  key 
people  within  its  IT  investment  process,  (2)  delineate  the  significant 
events  and  decision  points  within  the  processes,  (3)  identify  the  external 
and  environmental  factors  that  will  influence  the  processes  (that  is,  legal 
constraints,  the  behavior  of  key  subordinate  agencies  and  military 
customers,  and  the  practices  of  commercial  logistics  that  DLA  is  trying  to 
emulate  as  part  of  DLA  21);  and  (4)  explain  how  IT  investment-related 
processes  will  be  coordinated  with  other  organizational  plans  and 
processes.  DLA  does  not  have  guidance  that  sufficiently  addresses  these 
issues.  Policies  and  procedures  governing  operations  are  in  draft  for  one 
board  and  have  not  been  developed  for  the  two  other  boards.  Without  this 
guidance  governing  the  operations  of  the  investment  boards,  the  agency  is 


10We  did  not  address  the  alignment  of  these  boards,  as  this  is  a  stage  3  critical  process. 

“Because  the  DLA  Investment  Council  was  only  recently  established  and  is  still  defining  its 
role,  we  did  not  talk  to  members  about  their  implementation  of  stage  2  critical  processes. 
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at  risk  of  performing  key  investment  decisionmaking  activities 
inconsistently.  Such  guidance  would  also  provide  a  degree  of  transparency 
that  is  helpful  in  both  communicating  and  demonstrating  how  these 
decisions  are  made. 

Table  3  summarizes  the  ratings  for  each  key  practice  and  the  specific 
findings  supporting  the  ratings. 

Table  3:  IT  Investment  Board  Operation 

Type  of  process 

Process 

Rating 

Summary  of  evidence 

Organizational 

commitment 

1 .  Organization-specific  IT 
investment  process  guidance  is 
created  to  direct  each  board’s 
operations. 

Not  executed 

Policies  and  procedures  governing  operations  are  in  draft 
for  one  board  and  have  not  been  developed  for  the  two 
other  boards. 

2.  Organization  executives  and 
line  managers  support  and  carry 
out  IT  investment  board  decisions. 

Executed 

According  to  IT  investment  officials,  project  managers, 
and  agency  documentation,  the  boards  have  a  process 
for  ensuring  that  organization  executives  and  line 
managers  support  and  carry  out  board  decisions.  This 
process  (which  is  largely  undocumented)  involves 
capturing  these  decisions  in  meeting  minutes,  assigning 
staff  to  carry  out  the  decisions,  and  tracking  the  actions 
on  a  reqular  basis  until  the  issues  are  addressed. 

Prerequisites 

1.  Adequate  resources  are 
provided  for  operating  each  IT 
investment  board. 

Executed 

According  to  board  members,  the  boards  have  adequate 
resources  to  operate. 

2.  Board  members  understand  the 
investment  board’s  policies  and 
procedures  and  exhibit  core 
competencies  in  using  the  IT 
investment  approach  via  training, 
education,  or  experience. 

Executed 

Board  members  informed  us  of  their  understanding  of 
their  board’s  informal,  largely  undocumented  practices. 
Members  had  experience  in  making  investment  decisions 
and  had  knowledge  in  areas  such  as  capital  budgeting 
methods  and  economic  evaluation  techniques. 

Activities 

1 .  Each  investment  board  is 
created  and  defined  with  board 
membership  integrating  both  IT 
and  business  knowledge. 

Executed 

Membership  of  the  oversight  boards  performing 
investment  board  functions  integrates  IT  and  business 
knowledge. 

2.  Each  IT  investment  board 
operates  according  to  written 
policies  and  procedures  in 
organization-specific  IT  investment 
process  guidance. 

Not  executed 

Policies  and  procedures  governing  operations  are  in  draft 
for  one  board  and  have  not  been  developed  for  the  two 
other  boards  (see  organizational  commitment  1). 

Source:  GAO. 


IT  Project  Inventory 
Established,  but  Not 
Maintained  According  to 
Policies  and  Procedures 


An  IT  project  inventory  provides  information  to  investment  decision¬ 
makers  to  help  evaluate  the  impacts  and  opportunities  created  by 
proposed  or  continuing  investments.  This  inventory  (which  can  take  many 
forms)  should,  at  a  minimum,  identify  the  organization’s  IT  projects 
(including  new  and  existing  systems)  and  a  defined  set  of  relevant 
investment  management  information  about  them  (for  example,  purpose, 
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owner,  lifecycle  stage,  budget  cost,  physical  location,  and  interfaces  with 
other  systems).  Information  from  the  IT  project  inventory  can,  for 
example,  help  identify  systems  across  the  organization  that  provide  similar 
functions  and  help  avoid  the  commitment  of  additional  funds  for 
redundant  systems  and  processes.  It  can  also  help  determine  more  precise 
development  and  enhancement  costs  by  informing  decisionmakers  and 
other  managers  of  interdependencies  among  systems  and  how  potential 
changes  in  one  system  can  affect  the  performance  of  other  systems. 

According  to  ITTM,  effectively  managing  an  IT  project  inventory  requires, 
among  other  things,  (1)  identifying  IT  projects,  collecting  relevant 
information  about  them,  and  capturing  this  information  in  a  repository, 

(2)  assigning  responsibility  for  managing  the  IT  project  inventory  process 
to  ensure  that  the  inventory  meets  the  needs  of  the  investment 
management  process,  (3)  developing  written  policies  and  procedures  for 
maintaining  the  IT  project  inventory,  (4)  making  information  from  the 
inventory  available  to  staff  and  managers  throughout  the  organization  so 
they  can  use  it,  for  example,  to  build  business  cases  and  to  support  project 
selection  and  control  activities,  and  (5)  maintaining  the  IT  project 
inventory  and  its  information  records  to  contribute  to  future  investment 
selections  and  assessments.  (The  full  list  of  key  practices  is  provided  in 
table  4.) 

DLA  has  executed  many  of  the  key  practices  in  this  critical  process.  For 
example,  according  to  DLA’s  CIO,  IT  projects  are  identified  and  specific 
information  about  them  is  entered  into  a  central  repository  called  the  DLA 
Profile  System  (DPS).  DPS  includes,  among  other  things,  project 
descriptions,  key  contact  information,  lifecycle  stage,  and  system 
interfaces.  In  addition,  the  CIO  is  responsible  for  managing  the  IT  project 
identification  process  to  ensure  that  DPS  meets  the  needs  of  the 
investment  management  process.  However,  DLA  has  not  defined  written 
policies  and  procedures  for  how  and  when  users  should  add  to  or  update 
information  in  the  DPS.  In  addition,  DLA  is  not  maintaining  DPS  records, 
which  would  be  useful  during  future  project  selections  and  investment 
evaluations,  and  for  documenting  the  evolution  of  a  project’s  development. 
Without  appropriate  policies  and  procedures  in  place  to  describe  the 
objectives  and  information  requirements  of  the  inventory,  DPS  is  not  being 
maximized  as  an  effective  tool  to  assist  in  the  fundamental  analysis 
essential  to  effective  decisionmaking. 

Table  4  summarizes  the  ratings  for  each  key  practice  and  the  specific 
findings  supporting  the  ratings. 


Page  14 


GAO-02 -3 14  DLA's  Investment  Management  Process 


Table  4:  IT  Project  Identification 

Type  of  process 

Process 

Rating 

Summary  of  evidence 

Organizational 

commitment 

1 .  The  organization  has  written  policies 
and  procedures  for  identifying  its  IT 
projects  and  collecting  in  an  inventory 
the  information  about  the  IT  projects 
that  is  relevant  to  the  investment 
management  process. 

Not 

executed 

DLA  does  not  have  written  policies  and  procedures  for 
identifying  its  IT  projects  or  for  collecting  in  an  inventory 
the  information  about  the  IT  projects  that  is  relevant  to  the 
investment  management  process. 

2.  An  official  is  responsible  for 
managing  the  IT  project  identification 
process  to  ensure  that  the  inventory 
meets  the  needs  of  the  investment 
management  process. 

Executed 

The  CIO  is  responsible  for  managing  the  IT  project 
identification  process  to  ensure  that  the  inventory  meets 
the  needs  of  the  investment  management  process. 

Prerequisites 

1 .  Adequate  resources  are  provided  for 
identifying  IT  projects  and  collecting 
relevant  information  into  an  inventory. 

Executed 

According  to  the  officials  responsible  for  managing  DPS, 
adequate  resources  are  provided  for  identifying  IT  projects 
and  collecting  relevant  information  into  an  inventory. 

Activities 

1 .  The  organization’s  IT  projects  are 
identified  and  specific  information 
about  them  is  collected  in  an  inventory. 

Executed 

According  to  the  CIO  and  our  review  of  the  inventory, 

DLA’s  IT  projects  are  identified  and  specific  information 
about  them  is  collected  in  the  DLA  Profile  System  (DPS). 

2.  Changes  to  IT  projects  are 
identified,  and  change  information  is 
collected  in  the  inventory. 

Executed 

According  to  the  CIO  and  our  review  of  the  inventory,  IT 
projects  changes  are  identified,  and  change  information  is 
collected  in  the  inventory. 

3.  Information  from  the  inventory  is 
available  on  demand  to  decision¬ 
makers  and  other  affected  parties. 

Executed 

To  ensure  that  the  information  from  the  inventory  is 
available  to  decisionmakers  and  other  affected  parties, 

DLA  has  placed  DPS  on  its  intranet. 

4.  The  IT  project  inventory  and  its 
information  records  are  maintained  to 
contribute  to  future  investment 
selections  and  assessments. 

Not 

executed 

DPS  records  are  not  maintained  to  contribute  to  future 
investment  selections  and  assessments. 

Source:  GAO. 


Key  Project  Oversight  Practices  Investment  review  boards  should  effectively  oversee  IT  projects 
Occurring,  but  IT  Inventory  throughout  all  lifecycle  phases  (concept,  design,  development,  testing, 

Not  Being  Used  implementation,  and  operations/maintenance).  At  stage  2  maturity, 

investment  review  boards  should  review  each  project’s  progress  toward 
predefined  cost  and  schedule  expectations,  using  established  criteria  and 
performance  measures,  and  should  take  corrective  actions  to  address  cost 
and  milestone  variances. 

According  to  ITIM,  effective  project  oversight  requires,  among  other 
things,  (1)  having  written  polices  and  procedures  for  project  management, 
(2)  developing  and  maintaining  an  approved  management  plan  for  each  IT 
project,  (3)  having  written  policies  and  procedures  for  oversight  of  IT 
projects,  (4)  making  up-to-date  cost  and  schedule  data  for  each  project 
available  to  the  oversight  boards,  (5)  reviewing  each  project’s 
performance  by  regularly  comparing  actual  cost  and  schedule  data  to 
expectations,  (6)  ensuring  that  corrective  actions  for  each  under- 
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performing  project  are  documented,  agreed  to,  implemented,  and  tracked 
until  the  desired  outcome  is  achieved,  and  (7)  using  information  from  the 
IT  project  inventory.  (The  complete  list  of  key  practices  is  provided  in 
table  5.) 

DLA  has  executed  most  of  the  key  practices  in  this  area.  In  particular,  DLA 
relies  on  the  guidance  in  the  Department  of  Defense  5000  series  directives 
for  project  management  and  draft  guidance  in  an  Automated  Information 
System  (AIS)  Emerging  Program  Life-Cycle  Management  (LCM)  Review 
and  Milestone  Approval  Directive  for  specific  IT  project  management.  In 
addition,  for  each  of  the  four  projects  we  reviewed,  a  project  management 
plan  had  been  approved,  and  cost  and  schedule  controls  were  addressed 
during  project  review  meetings.  Further,  based  on  our  review  of  project 
documentation  and  in  discussion  with  project  managers,  up-to-date  cost 
and  schedule  project  data  were  provided  to  the  PEO  Review  Board.  This 
board  oversees  project  performance  regularly  by  comparing  actual  cost 
and  schedule  data  to  expectations  and  has  a  process  for  ensuring  that,  for 
underperforming  projects,  corrective  actions  are  documented,  agreed  to, 
and  tracked. 

Notwithstanding  these  strengths,  DLA  has  some  weaknesses  in  project 
oversight.  Specifically,  although  the  Corporate  Board  and  the  Investment 
Council  have  written  charters,  there  are  no  written  policies  or  procedures 
that  define  their  role  in  collectively  overseeing  IT  projects.  Without  these 
policies  and  procedures,  project  oversight  may  be  inconsistently  applied, 
leading  to  the  risk  that  performance  problems,  such  as  cost  overruns  and 
schedule  slippages,  may  not  be  identified  and  resolved  in  a  timely  manner. 

In  addition,  according  to  representatives  from  the  oversight  boards,  they 
do  not  use  information  from  the  IT  project  inventory  to  oversee  projects 
because  they  are  more  comfortable  using  more  traditional  methods  of 
obtaining  and  using  information  (that  is,  informally  talking  with  subject 
matter  experts  and  relying  on  experience).  The  inventory  is  of  value  only 
to  the  extent  that  decisionmakers  use  it.  As  discussed  earlier,  while  the 
inventory  need  not  be  the  only  source  of  information,  it  should 
nevertheless  serve  as  a  reliable  and  consistent  tool  for  understanding 
project  and  overall  portfolio  decisions. 

Table  5  summarizes  the  ratings  for  each  key  practice  and  the  specific 
findings  supporting  the  ratings. 
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Table  5:  IT  Project  Oversight 

Type  of 
process 

Process 

Rating 

Summary  of  evidence 

Organizational 

commitment 

1 .  The  organization  has  written 
policies  and  procedures  for  project 
management. 

Executed 

DOD’s  5000  series  and  DLA’s  draft  Automated  Information 
System  (AIS)  Emerging  Program  Life-cycle  Management 
(LCM)  Process,  Review,  and  Milestone  Approval  Directive 
define  DLA  policies  and  procedures  for  project  management. 

2.  The  organization  has  written 
policies  and  procedures  for 
management  oversight  of  IT 
projects. 

Not  executed 

DLA  does  not  have  written  policies  and  procedures  for  the 
Investment  Council  or  for  the  Corporate  Board’s  oversight  of 

IT  projects.  DLA  has  draft  policies  and  procedures  for  the 

PEO  Review  Board’s  oversiqht  of  IT  projects. 

Prerequisites 

1 .  Adequate  resources  are 
provided  to  assist  the  board (s)  in 
overseeing  IT  projects. 

Executed 

According  to  board  members,  adequate  resources  are 
provided  for  overseeing  IT  projects. 

2.  Each  IT  project  has  and 
maintains  an  approved  project 
management  plan  that  includes 
cost  and  schedule  controls. 

Executed 

DOD  and  DLA  directives  require  that  each  project  have  an 
updated  and  approved  project  management  plan.  In  addition, 
cost  and  schedule  controls  are  addressed  during  project 
review  meetings.  The  four  projects  we  reviewed  each  have 
an  approved  project  management  plan.  Cost  and  schedule 
controls  are  also  addressed  during  review  meetings  for  these 
projects. 

3.  An  IT  investment  board  is 
operating. 

Executed 

DLA  has  a  number  of  oversight  boards  performing  IT 
investment  management  functions.  Those  responsible  for 
making  decisions  regarding  major  investments  are  the 
Corporate  Board,  the  DLA  Investment  Council,  and  the  PEO 
Review  Board. 

4.  Information  from  the  IT  projects 
inventory  is  used  by  the  IT 
investment  board  as  applicable. 

Not  executed 

According  to  members  of  the  oversight  boards,  they  do  not 
use  information  from  the  IT  project  inventory  to  oversee 
projects 

Activities 

1 .  Each  project’s  up-to-date  cost 
and  schedule  data  are  provided  to 
the  appropriate  IT  investment 
board. 

Executed 

The  draft  Automated  Information  System  (AIS)  Emerging 
Program  Life-cycle  Management  (LCM)  Process,  Review, 
and  Milestone  Approval  Directive  requires  that  information  be 
provided  to  oversight  entities.  Based  on  our  review  of  project 
documentation  and  in  discussion  with  project  managers,  up- 
to-date  cost  and  schedule  project  data  are  provided  to 
appropriate  oversiqht  boards. 

2.  Using  established  criteria,  the  IT 
investment  board  oversees  each  IT 
project’s  performance  monthly  and 
quarterly  by  comparing  actual  cost 
and  schedule  data  to  expectations. 

Executed 

Based  on  our  review  of  status  briefings,  the  PEO  Review 
Board,  which  is  responsible  for  overseeing  IT  project 
performance,  does  so  regularly  by  comparing  actual  cost  and 
schedule  data  to  expectations  using  established  criteria. 

3.  The  IT  investment  board 
performs  special  reviews  of 
projects  that  have  not  met 
predetermined  performance 
standards. 

Executed 

DOD  directives  and  DLA’s  draft  Automated  Information 

System  (AIS)  Emerging  Program  Life-cycle  Management 
(LCM)  Process,  Review,  and  Milestone  Approval  Directive 
specify  when  special  reviews  are  to  occur.  According  to  IT 
investment  officials,  DLA  has  not  had  to  perform  any  of  these 

reviews.  Based  on  our  review  of  project  documentation, 
special  reviews  were  not  necessary  for  the  four  projects  we 
reviewed. 
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Type  of 
process 

Process 

Rating 

Summary  of  evidence 

4.  Appropriate  corrective  actions 
for  each  underperforming  project 
are  defined,  documented,  and 
agreed  to  by  the  IT  investment 
board  and  the  project  manager. 

Executed 

According  to  IT  investment  officials  and  agency 
documentation,  DLA  has  a  process  for  ensuring  that 
corrective  actions  for  underperforming  projects  are  defined, 
documented,  and  agreed  to  by  the  IT  investment  board  and 
the  project  manaqer. 

5.  Corrective  actions  are 
implemented  and  tracked  until  the 
desired  outcome  is  achieved. 

Executed 

According  to  IT  investment  officials  and  agency 
documentation,  DLA  has  a  process  for  ensuring  that 
corrective  actions  are  implemented  and  tracked  until  the 
desired  outcome  is  achieved. 

Source:  GAO. 


Practices  to  Identify  Business  Defining  business  needs  for  each  IT  project  helps  ensure  that  projects 

Needs  Are  Occurring  with  support  the  organization’s  mission  goals  and  meets  users’  needs.  This 

Most,  but  Not  All,  Projects  critical  process  creates  the  link  between  the  organization’s  business 

objectives  and  its  IT  management  strategy.  According  to  ITIM,  effectively 
identifying  business  needs  requires,  among  other  things,  (1)  defining  the 
organization’s  business  needs  or  stated  mission  goals,  (2)  identifying  users 
for  each  project  who  will  participate  in  the  project’s  development  and 
implementation,  (3)  training  IT  staff  adequately  in  identifying  business 
needs,  and  (4)  defining  business  needs  for  each  project.  (The  complete  list 
of  key  practices  is  provided  in  table  6.) 

DLA  has  executed  all  but  one  of  the  key  practices  associated  with 
effectively  defining  business  needs  for  IT  projects.  For  example,  DLA’s 
mission  goals  are  described  in  DLA’s  strategic  plan.  In  addition,  according 
to  IT  investment  management  officials,  the  IT  staff  is  adequately  trained  in 
identifying  business  needs  because  they  generally  have  prior  functional 
unit  experience.  In  addition,  according  to  DLA  directives,  IT  projects  are 
assigned  an  Integrated  Process  Team  (IPT)  to  guide  and  direct  the  project 
through  the  development  lifecycle.  The  IPTs  are  composed  of  IT  and 
functional  staff.  Moreover,  DOD  and  DLA  directives  require  that  business 
requirements  and  system  users  be  identified  and  that  users  participate  in 
the  lifecycle  management  of  the  project.  According  to  an  IT  investment 
official,  each  IT  project  has  a  users’  group  that  meets  throughout  the 
lifecycle  to  discuss  problems  and  potential  changes  related  to  the  system. 
We  verified  that  this  was  the  case  for  the  four  projects  we  reviewed. 

While  the  business  needs  for  three  of  the  four  projects  we  reviewed  were 
clearly  identified  and  defined, 12  DLA  has  reported  that  this  has  not  been 


12DLA  has  determined  that  the  fourth  project  we  reviewed — SAMMS — is  no  longer  meeting 
the  agency’s  business  needs. 
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consistently  done  for  all  IT  projects.  According  to  IT  investment 
management  officials,  this  inconsistency  arose  because  policies  and 
procedures  for  developing  business  needs  were  not  always  followed  or 
required. 

DLA  officials  have  stated  that  they  are  developing  new  guidance  to 
address  this  problem.  However,  until  this  guidance  is  implemented  and 
enforced,  DLA  cannot  effectively  demonstrate  that  priority  mission  and 
business  improvement  needs  are  forming  the  basis  for  all  its  IT  investment 
decisions. 

Table  6  summarizes  the  ratings  for  each  key  practice  and  the  specific 
findings  supporting  the  ratings. 

Table  6:  Business  Needs  Identification  for  IT  Projects 

Type  of  process 

Process 

Rating 

Summary  of  evidence 

Organizational 

commitment 

1 .  The  organization  has  written  policies 
and  procedures  for  identifying  the 
business  needs  (and  the  associated 
users)  of  each  IT  project. 

Executed 

DOD  and  DLA  directives  include  policies  and  procedures 
for  identifying  the  business  needs  and  associated  users 
of  IT  systems. 

Prerequisites 

1.  Adequate  resources  are  provided  for 
identifying  business  needs  and 
associated  users. 

Executed 

According  to  an  IT  investment  management  official, 
adequate  resources  exist  to  identify  business  needs  and 
associated  users. 

2.  The  organization  has  defined 
business  needs  or  stated  mission 
qoals. 

Executed 

DLA’s  mission  goals  are  stated  in  its  Strategic  Plan 

2000,  generally  known  as  DLA  21 . 

3.  IT  staff  are  trained  in  business  needs 
identification. 

Executed 

According  to  IT  investment  management  officials  and 
representatives  from  the  oversight  boards,  IT  staff 
generally  come  from  the  business  units  and  are 
therefore  adequately  trained  in  business  needs 
identification. 

4.  IT  projects  are  identified  and  specific 
information  about  them  is  collected  in 
an  inventory. 

Executed 

According  to  the  CIO,  IT  projects  in  DLA  are  identified  in 
DPS,  which  is  DLA’s  IT  project  inventory.  We  verified 
that  all  of  DLA’s  systems  in  development  and  the  four 
projects  we  reviewed  were  included  in  the  DPS. 
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Type  of  process 

Process 

Rating 

Summary  of  evidence 

Activities 

1 .  The  business  needs  for  each  IT 
project  are  clearly  identified  and 
defined. 

Not 

executed 

DOD  and  DLA  directives  require  that  business  needs  for 
each  IT  project  be  specified  in  the  mission  needs 
statement.  We  verified  that  business  needs  for  three  of 
the  four  projects  we  reviewed  were  clearly  identified  and 
defined.3  However,  DLA  has  reported  that  the  business 
needs  for  each  IT  project  are  not  clearly  identified  and 
defined,  because  the  policies  and  procedures  are  not 
always  followed  or  required. 

2.  Specific  users  are  identified  for  each 
IT  project. 

Executed 

According  to  IT  investment  officials,  specific  users  are 
identified  for  each  IT  project.  We  verified  that  specific 
users  were  identified  for  the  four  projects  we  reviewed. 

3.  Identified  users  participate  in  project 
management  throughout  a  project’s 
lifecycle. 

Executed 

According  to  an  IT  investment  official,  each  IT  project 
has  a  users’  group  that  meets  throughout  the  lifecycle  to 
discuss  problems  and  potential  changes  related  to  the 
system.  We  verified  that  this  was  the  case  for  the  four 
projects  we  reviewed. 

aDLA  has  determined  that  the  fourth  project  we  reviewed — SAMMS — is  no  longer  meeting  the 
agency’s  business  needs. 


Source:  GAO. 


IT  Investment  Selection  Selecting  new  IT  proposals  requires  an  established  and  structured  process 

Process  Not  Yet  Established  to  ensure  informed  decisionmaking  and  infuse  management 

accountability.  According  to  ITIM,  this  critical  process  requires,  among 
other  things,  (1)  making  funding  decisions  for  new  IT  proposals  according 
to  an  established  process,  (2)  providing  adequate  resources  for  proposal 
selection  activities,  (3)  using  an  established  proposal  selection  process, 

(4)  analyzing  and  ranking  new  IT  proposals  according  to  established 
selection  criteria,  including  cost  and  schedule  criteria,  and  (5)  designating 
an  official  to  manage  the  proposal  selection  process.  (The  complete  list  of 
key  practices  is  provided  in  table  7.) 

DLA  has  executed  some  of  the  key  practices  for  investment  proposal 
selection.  For  example,  DLA  executives  make  funding  decisions  for  IT 
investments  using  DOD’s  Program  Objective  Memorandum  (POM) 
process,13  which  is  part  of  DOD’s  annual  budgeting  process.  Through  this 
process,  proposals  for  new  projects  or  enhancements  to  ongoing  projects 
are  evaluated  by  DLA’s  IT  and  financial  groups  and  submitted  to  OSD 
through  DLA’s  Corporate  Board  with  recommendations  for  funding 
approval.  In  addition,  according  to  the  CIO,  adequate  resources  have  been 
provided  to  carry  out  activities  related  to  the  POM  process. 


13The  POM  serves  as  the  basis  for  the  services’  budget  estimates.  It  shows  program  needs 
for  at  least  the  next  5  years  and  includes  cost  estimates  of  manpower,  force  levels, 
procurement,  facilities,  and  research  and  development. 
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Nonetheless,  DLA  has  yet  to  execute  some  of  the  critical  practices  related 
to  this  process  area.  Specifically,  DLA  acknowledges  that  the  agency  is  not 
analyzing  and  prioritizing  new  IT  proposals  according  to  established 
selection  criteria.  Instead,  the  Corporate  Board  uses  the  expertise  from  the 
IT  organization  and  its  own  judgment  to  analyze  and  prioritize  projects.  To 
its  credit,  DLA  recognizes  that  it  cannot  continue  to  rely  solely  on  the  POM 
process  to  make  sound  IT  investment  selection  decisions.  Therefore,  the 
agency  has  been  working  to  establish  an  IT  selection  process  over  the  past 
two  budget  cycles  that  is  more  investment-focused  and  includes  increased 
involvement  from  IT  Operations  staff,  necessary  information,  and 
established  selection  criteria. 

Until  DLA  implements  an  effective  IT  investment  selection  process  that  is 
well  established  and  understood  throughout  the  agency,  executives  cannot 
be  adequately  assured  that  they  are  consistently  and  objectively  selecting 
proposals  that  best  meet  the  needs  and  priorities  of  the  agency. 

Table  7  summarizes  the  ratings  for  each  key  practice  and  the  specific 
findings  supporting  the  ratings. 


Table  7:  Proposal  Selection 

Type  of  process  Process 

Rating 

Summary  of  evidence 

Organizational  1 .  Executives  and  managers  follow  an  Not 

commitment  established  selection  process.  executed 


because  the  current  POM-driven  process  does  not  satisfy 
the  ITIM  requirements  for  an  effective  IT  investment 
selection  process.  DLA  has  been  instituting  a  selection 
process  over  the  past  two  POM  cycles  to  include  increased 
involvement  from  IT  Operations  staff,  additional 
documentation  requirements,  and  established  selection 
criteria.  This  new  selection  process  is  not  fully 


2.  An  official  is  designated  to  manage 
the  proposal  selection  process. 

Executed 

The  PEO  is  responsible  for  managing  the  proposal 
selection  process. 

Prerequisites 

1.  Adequate  resources  are  provided 
for  proposal  selection  activities. 

Executed 

The  CIO  has  stated  that  DLA  has  adequate  resources  for 
activities  related  to  the  POM  process. 

Activities 

1 .  The  organization  uses  a  structured 
process  to  develop  new  IT  proposals. 

Executed 

DOD’s  5000  and  DLA  directives  delineate  a  process  for 
developing  new  IT  proposals. 

2.  Executives  analyze  and  prioritize 
new  IT  proposals  according  to 
established  selection  criteria. 

Not 

executed 

DLA  does  not  have  any  established  and  documented 
criteria  for  analyzing  and  prioritizing  new  IT  proposals. 
According  to  a  Corporate  Board  member,  they  rely  on  IT 
Operations  expertise  and  their  own  judgment  to  analyze 
and  prioritize  projects. 

3.  Executives  make  funding  decisions 
for  new  IT  proposals  according  to  an 
established  process. 

Executed 

DLA  uses  DOD’s  Program  Objective  Memorandum  process 
to  make  funding  decisions  on  new  IT  proposals. 

Source:  GAO. 
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IT  Investments  Are  Not 
Being  Managed  as  a 
Complete  Portfolio 


An  IT  investment  portfolio  is  an  integrated,  enterprisewide  collection  of 
investments  that  are  assessed  and  managed  collectively  based  on  common 
criteria.  Managing  investments  within  the  context  of  such  a  portfolio  is  a 
conscious,  continuous,  and  proactive  approach  to  expending  limited 
resources  on  an  organization’s  competing  initiatives  in  light  of  the  relative 
benefits  expected  from  these  investments.  Taking  an  enterprisewide 
perspective  enables  an  organization  to  consider  its  investments 
comprehensively  so  that  the  collective  investments  optimally  address  its 
mission,  strategic  goals,  and  objectives.  This  portfolio  approach  also 
allows  an  organization  to  determine  priorities  and  make  decisions  about 
which  projects  to  fund  based  on  analyses  of  the  relative  organizational 
value  and  risks  of  all  projects,  including  projects  that  are  proposed,  under 
development,  and  in  operation. 

According  to  ITEM,  stage  3  maturity  includes  (1)  defining  portfolio 
selection  criteria,  (2)  engaging  in  project-level  investment  analysis, 

(3)  developing  a  complete  portfolio  based  on  the  investment  analysis, 

(4)  maintaining  oversight  over  the  investment  performance  of  the 
portfolio,  and  (5)  aligning  the  authority  of  IT  investment  boards. 

Table  8  describes  the  purposes  for  the  critical  processes  in  stage  3. 


Table  8:  Stage  3  Critical  Processes 

Critical  process 

Purpose 

Authority  alignment  of  IT 
investment  boards 

To  ensure  that  IT  investments  are  selected  and 
managed  by  the  appropriate  investment  board. 

Portfolio  selection  criteria 
definition 

To  ensure  that  the  organization  develops  and  maintains 
IT  portfolio  selection  criteria  that  support  its  mission, 
organizational  strategies,  and  business  priorities. 

Investment  analysis 

To  ensure  that  all  IT  investments  are  consistently 
analyzed  and  prioritized  according  to  the  organization’s 
portfolio  selection  criteria. 

Portfolio  development 

To  ensure  that  an  optimal  IT  investment  portfolio  with 
manageable  risks  and  returns  is  selected  and  funded. 

Portfolio  performance 
oversight 

To  ensure  that  each  IT  investment  portfolio  achieves  its 
cost,  benefit,  schedule,  and  risk  expectations. 

Source:  GAO. 


According  to  DLA  officials,  they  are  currently  focusing  on  implementing 
stage  2  processes  and  have  not  implemented  any  of  the  critical  processes 
in  stage  3.  Until  the  agency  fully  implements  both  stage  2  and  3  processes, 
it  cannot  consider  investments  in  a  comprehensive  manner  and  determine 
whether  it  has  the  appropriate  mix  of  IT  investments  to  best  meet  its 
mission  needs  and  priorities. 
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DLA  Lacks  a  Plan  to 
Guide  Improvement 
Efforts 


DLA  recognizes  the  need  to  improve  its  IT  investment  processes,  but  it  has 
not  yet  developed  a  plan  for  systematically  correcting  weaknesses.  To 
properly  focus  and  target  IT  investment  process  improvements,  an 
organization  should  fully  identify  and  assess  current  process  strengths  and 
weaknesses  (that  is,  create  an  investment  management  capability 
baseline)  as  the  first  step  in  developing  and  implementing  an  improvement 
plan.  As  we  have  previously  reported,14  this  plan  should,  at  a  minimum, 

(1)  specify  measurable  goals,  objectives,  milestones,  and  needed 
resources,  and  (2)  clearly  assign  responsibility  and  accountability  for 
accomplishing  well-defined  tasks.  The  plan  should  also  be  documented 
and  approved  by  agency  leadership.  In  implementing  the  plan,  it  is 
important  that  DLA  measure  and  report  progress  against  planned 
commitments,  and  that  appropriate  corrective  action  be  taken  to  address 
deviations. 


DLA  does  not  have  such  a  plan.  In  March  2001,  it  attempted  to  baseline 
agency  IT  operations  by  reviewing  its  project-level  investment 
management  practices  using  ITIM.  This  effort  identified  practice  strengths 
and  weaknesses,  but  DLA  considered  the  assessment  to  be  preliminary  (to 
be  followed  by  a  more  comprehensive  assessment  at  an  unspecified  later 
date)  and  limited  in  scope.  DLA  used  the  assessment  results  to  establish 
broad  milestones  for  strengthening  its  investment  management  process. 
The  agency  did  not,  however,  develop  a  complete  process  improvement 
plan.  For  example,  it  did  not  (1)  specify  required  resources  to  accomplish 
the  various  tasks,  (2)  clearly  assign  responsibility  and  accountability  for 
accomplishing  the  tasks,  (3)  obtain  support  from  senior  level  officials,  and 
(4)  establish  performance  measures  to  evaluate  the  effectiveness  of  the 
completed  tasks.  At  the  same  time,  the  agency  has  separately  begun  other 
initiatives  to  improve  its  investment  management  processes,  but  these 
initiatives  are  not  aligned  with  the  established  milestones  or  with  each 
other.  The  DLA  CIO  characterizes  the  agency’s  approach  to  its  various 
process  improvement  efforts  as  a  necessary  progression  that  includes 
some  inevitable  “trial  and  error”  as  it  moves  toward  a  complete  process 
improvement  plan. 

Without  such  a  plan  that  allows  the  agency  to  systematically  prioritize, 
sequence,  and  evaluate  improvement  efforts,  DLA  jeopardizes  its  ability  to 


14U.S.  General  Accounting  Office,  Air  Traffic  Control:  Immature  Software  Acquisition 
Processes  Increase  FAA  System  Acquisition  Risks,  GAO/AIMD-97-47  (Washington,  D.C.: 
March  21, 1997). 
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establish  a  mature  investment  process  that  includes  selection  and  control 
capabilities  that  result  in  greater  certainty  about  future  IT  investment 
outcomes. 


Conclusions 


Until  recently,  IT  investment  management  has  not  been  an  area  of  DLA 
management  attention  and  focus.  As  a  result,  DLA  currently  finds  itself 
without  some  of  the  capabilities  that  it  needs  to  ensure  that  its  mix  of  IT 
investments  best  meets  the  agency’s  mission  and  business  priorities.  To  its 
credit,  DLA  now  recognizes  the  need  to  strengthen  its  IT  investment 
management  and  has  taken  positive  steps  to  begin  doing  so.  However, 
several  critical  IT  investment  management  capabilities  need  to  be 
enhanced  before  DLA  can  have  reasonable  assurance  that  it  is  maximizing 
the  value  of  its  IT  investment  dollar  and  minimizing  the  associated  risks. 
Moreover,  DLA  does  not  yet  have  a  process  improvement  plan  that  is 
endorsed  and  supported  by  agency  leadership.  The  absence  of  such  a  plan 
limits  DIA’s  prospects  for  introducing  the  management  capabilities 
necessary  for  making  prudent  decisions  that  maximize  the  benefits  and 
minimize  the  risks  of  its  IT  investment. 


Recommendations 


To  strengthen  DLA’s  investment  management  capability  and  address  the 
weaknesses  discussed  in  this  report,  we  recommend  that  the  secretary  of 
defense  direct  the  DLA  director  to  designate  the  development  and 
implementation  of  effective  IT  investment  management  processes  as  an 
agencywide  priority. 

Further,  we  recommend  that  the  secretary  of  defense  have  the  DLA 
director  do  the  following: 

Develop  a  plan,  within  6  months,  for  implementing  IT  investment 
management  process  improvements  that  is  based  on  GAO’s  ITIM  stage  2 
and  3  critical  processes. 

Ensure  that  the  plan  specifies  measurable  goals  and  time  frames,  defines  a 
management  structure  for  directing  and  controlling  the  improvements,  and 
establishes  review  milestones. 

Ensure  that  the  plan  focuses  first  on  correcting  the  weakness  in  the  ITIM 
stage  2  critical  processes,  because  these  processes  collectively  provide  the 
foundation  for  building  a  mature  IT  investment  management  process. 
Specifically: 

•  Develop  and  issue  guidance  covering  the  scope  and  operations  of 
DLA’s  investment  review  boards.  Such  guidance  should  include,  at  a 
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minimum,  specific  definitions  of  the  roles  and  responsibilities  within 
the  IT  investment  process;  an  outline  of  the  significant  events  and 
decision  points  within  the  processes;  an  identification  of  the  external 
and  environmental  factors  that  will  influence  the  processes  (for 
example,  legal  constraints,  the  behavior  of  key  suppliers  or  customers, 
or  industry  norms),  and  the  manner  in  which  IT  investment-related 
processes  will  be  coordinated  with  other  organization  plans  and 
processes. 

•  Develop  and  issue  policies  and  procedures  for  maintaining  DLA’s  IT 
projects  inventory  for  investment  management  purposes. 

•  Finalize  and  issue  policies  and  procedures  (including  the  use  of 
information  from  the  IT  systems  and  project  inventory)  for  the  PEO 
Review  Board’s  oversight  of  IT  projects. 

•  Develop  and  issue  similar  policies  and  procedures  for  the  other 
investment  boards. 

•  Finalize  and  issue  guidance  supporting  the  identification  of  business 
needs  and  implementing  management  controls  to  ensure  that  proposals 
submitted  to  DLA  for  review  clearly  identify  and  define  business 
requirements. 

•  Develop  and  issue  guidance  for  the  proposal  selection  process  in  such 
a  way  that  the  criteria  for  selection  are  clearly  set  forth,  including 
formally  assigning  responsibility  for  managing  the  proposal  selection 
process  and  establishing  management  controls  to  ensure  that  the 
proposal  selection  process  is  working  effectively. 

•  Ensure  that  the  plan  next  focuses  on  stage  3  critical  processes,  which  are 
necessary  for  portfolio  management,  because  along  with  the  stage  2 
foundational  processes,  these  processes  are  necessary  for  effective 
management  of  IT  investments. 

•  Implement  the  approved  plan  and  report  on  progress  made  against  the 
plan’s  goals  and  time  frames  to  the  secretary  of  defense  every  6  months. 


DOD  provided  what  it  termed  “official  oral  comments”  from  the  director 
for  acquisition  resources  and  analysis  on  a  draft  of  this  report.  In  its 
comments,  DOD  concurred  with  our  recommendations  and  described 
efforts  under  way  and  planned  to  implement  them.  However,  it 
recommended  that  two  report  captions  be  changed  to  more  accurately 
reflect,  in  DOD’s  view,  the  contents  of  the  report  and  to  eliminate  false 
impressions. 

Specifically,  DOD  recommended  that  we  change  one  caption  from  “DLA’s 
Capabilities  to  Effectively  Manage  IT  Investments  Are  Limited”  to  “DLA’s 
Capabilities  to  Effectively  Manage  IT  Investments  Should  Be  Improved.” 


Agency  Comments 
and  Our  Evaluation 


Page  25 


GAO-02-314  DLA's  Investment  Management  Process 


DOD  stated  that  this  change  is  needed  to  recognize  the  fact  that  DLA  has 
completed  about  75  percent  of  the  practices  associated  with  stage  2 
critical  processes.  We  do  not  agree.  As  stated  in  our  report,  to  effectively 
manage  IT  investments  an  agency  should  (1)  have  basic,  project-level 
control  and  selection  practices  in  place  (stage  2  processes)  and 

(2)  manage  its  projects  as  a  portfolio  of  investments  (stage  3  processes). 
Although  DLA  has  executed  most  of  the  key  practices  associated  with 
stage  2  processes,  the  agency  acknowledges  that  it  has  not  implemented 
any  of  the  stage  3  processes.  Therefore,  our  caption  as  written  describes 
DLA’s  IT  investment  management  capabilities  appropriately. 

In  addition,  DOD  recommended  that  we  change  the  caption  “DLA  Lacks  a 
Plan  to  Guide  Improvement  Efforts”  to  “DLA  Lacks  a  Published  Plan  to 
Guide  Improvement  Efforts.”  DOD  stated  that  this  change  is  needed 
because  DLA  has  developed  some  elements  of  an  implementation  plan.  We 
do  not  agree.  Our  point  is  that  DLA  did  not  have  a  complete  process 
improvement  plan,  not  that  it  has  yet  to  publish  the  plan  that  it  has.  As  we 
describe  in  the  report,  a  complete  plan  should,  at  a  minimum,  (1)  be  based 
on  a  full  assessment  of  process  strengths  and  weaknesses,  (2)  specify 
measurable  goals,  objectives,  milestones,  and  needed  resources, 

(3)  clearly  assign  responsibility  and  accountability  for  accomplishing  well- 
defined  tasks,  and  (4)  be  documented  and  approved  by  agency  leadership. 

In  contrast,  DLA’s  planning  document  was  based  on  a  preliminary 
assessment  of  only  stage  2  critical  processes  and  lacked  several  of  the 
critical  attributes  listed  above.  Moreover,  DOD  stated  in  its  comments  that 
DLA  has  not  completed  a  formally  documented  and  prioritized 
implementation  plan  to  resolve  stage  2  and  3  practice  weaknesses  and  has 
yet  to  complete  the  self-assessment  and  gap  analysis  necessary  to  define 
planned  action  items.  Accordingly,  it  is  clear  that  DLA  has  not  satisfied  the 
tenets  of  a  complete  plan,  and  thus  our  caption  is  accurate  as  written. 

DOD  provided  additional  comments  that  we  have  incorporated  as 
appropriate  in  the  report. 


We  are  sending  copies  of  this  report  to  the  chairmen  and  ranking  minority 
members  of  the  Subcommittee  on  Defense,  Senate  Committee  on 
Appropriations;  the  Subcommittee  on  Readiness  and  Management 
Support,  Senate  Committee  on  Armed  Services;  the  Subcommittee  on 
Defense,  House  Committee  on  Appropriations;  and  the  Subcommittee  on 
Military  Readiness,  House  Committee  on  Armed  Services.  We  are  also 
sending  copies  to  the  director,  Office  of  Management  and  Budget;  the 
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secretary  of  defense;  the  under  secretary  of  defense  for  acquisition, 
technology,  and  logistics;  the  deputy  under  secretary  of  defense  for 
logistics  and  materiel  readiness;  and  the  director,  Defense  Logistics 
Agency.  Copies  will  be  made  available  to  others  upon  request. 

If  you  have  any  questions  regarding  this  report,  please  contact  us  at 
(202)  512-3439  and  (202)  512-7351,  respectively,  or  by  e-mail  at 
hiter@gao.gov  and  mcclured@gao.gov.  An  additional  GAO  contact  and 
staff  acknowledgments  are  listed  in  appendix  II. 


Randolph  C.  Hite 

Director,  Information  Technology  Architecture 
and  Systems  Issues 


David  L.  McClure 

Director,  Information  Technology  Management  Issues 
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assistance  to  help  Congress  make  informed  oversight,  policy,  and  funding 
decisions.  GAO’s  commitment  to  good  government  is  reflected  in  its  core  values 
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